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ABSTRACT 



Previous research using various measures of teacher quality has largely failed to 
investigate the relationships among such assessment measures using multivariate 
procedures. Multivariate investigations are essential in studies of teacher competence 
since these methods view various competency variables in the larger contexts to which 
researchers design their results to generalize. The author demonstrates one approach to 
conducting such an investigation v^ith data from 496 student teachers using canonical 
correlation analysis. Measures of teacher luaowledge (subtests of the NTE and overall 
college GPA) served as predictor variables and measures of teaching performance 
(scores across several ratings on the Mississippi Teacher Assessment Instruments and 
final student teaching grade) served as criterion variables. The analysis yielded one 
statistically significant canonical root. Results suggest that the NTE subtest scores are 
not very good predictors of teaching performance, but that students' CPAs are relatively 
good predictors. 



PREDICTING TEACHING PERFORMANCE: A MULTIVARIATE INVESTIGATION 

In recent years, institutions of higher learning and state teacher credcntialing 
agencies have increasingly mandated several types of quantitative measures for 
prospective teachers in attempts to ensure the quality of teaching candidates. Pratt, 
Deiucia, and V illiams (1987) note that these measures have been particularly widely 
adopted among states in the southern United States. General tests of scholastic ability 
(e.g., the American College Test, the Scholastic Aptitude Test) have been used for initid 
screening of candidates prior to their entrance into college and early college success 
(usually measured by an overall college GPA) is often used to monitor entrance iato a 
teacher education program. Tests to assess the knowledge base more specifically related 
to teaching have also been used, with the various component tests of th« National 
Teacher Examinations (NTE) emerging as the most popularly used tests of this nature 
(Lines, 1985). In additior, measures of teaching performance have been utilized to 
quantitatively assess actually demonstrated teaching behaviors. 

With teachers receiving an increasingly large amcunt of public scrutiny, many 
feci that use of such measures to document the performance of teacher candidates is 
justifiable. For example, as Egan and Ferre (1989, p. 227) have noted, "facultj' 
members in teacher education programs need reliable information that can provide an 
accurate assessment of a student^s potential for success, that rs the ability [of the 
student] to complete educational requirements of a teacher education program and pass 
a competency examination.'' Similarly, Leiser (1981, p. 48) asserted, "a teacher 
[should] be able to demonstrate his or her competence in subject matter, in the English 
language, and in those other areas which every teacher ought to know before being 
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admitted to a classroom with a group of expectant children/' Moreover, various state- 
by-state content \. ''dity studies (e.g., Hankins & Hancock, 1984) indicate that tests such 
us the NTE are generally felt to be appropriate indicators of a common knowledge base 
for teachers (Rosner & Howey, 1982). However, Book and Freeman (1986) found that 
prospective teachers place a higher value on content-specific courses and on-the-job 
experience than on pedagogy-related courses as a trucial source of professional 
knowledge. 

Others doubt the usefulness of teacher testing, or at least recognize that testing is 
not a panacea for assuring teacher quality'. For instance, Lines (1985, p. 618) argued 
that tests of the teacher^s knowledge base 

. . . may measure the v/rong things, intro^'uce their own biases, or be 
otherwise poorly designed. They are also one-dimensional— measuring only 
knowledge and not other characteristics that make for an effective teacher, 
such as compassion, love of children, energy, ivisdom, dedication, and 
similar qualities. 

Similarly, Daniel, Sidors, and Slick (1991) offer evidence to suggest that typically used 
measures of teacher performance in the classroom are subject to rater bias. 
Overview of Previous Research on Teacher Competence Measures 

Previous researchers ha^ e focused en the relationships among the three 
previously-mentioned types of measures (i.e., measures of general academic competence, 
measures of a specific knowledge base for teachers, measures of teaching performance) 
as well as other indicators of competence of teacher candidates. Not surprisingly, the 
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results of these studies have prompted researchers (e.g., White & Tierney, 1989) to 
recommend a focus on multiple measures of competence in making teacher licensure 
decisions. 

Most previous studies have employed use of Pearson product-moment correlation 
coefficients or multiple regression procedures to determine the degree to which scores on 
one or more of the NTE subtests are related to various other measures of competence, 
with the iNTE subtests often serving as dependent variables. For instance, Hall (1964) 
found "reasonably high'* correlations between the original NTE's Weighted Common 
Exam scores and students' GPAs. Egan and Fesre (1989) found moderate to high 
correlations between students' GPAs and their scores on the professional knowledge, 
communication skills, and general knowledge NTE tests. White and Tierney (1989) 
found moderate relationships between Eiigllsh students' subject area GPAs and their 
scores on the English NTE specialty test. Erickson (1971) found a moder-ite negative 
correlation to exist between prospective English teachers' specialty area NTE and high 
school students' ratings of their student teaching performance on the Illinois Teacher 
Evaluation Questionnaire, suggesting that those with a more substantial, knowledge 
based were perceived by students to be less effective. Using multiple regression 
procedures, Olstad, Bcal, and Marrett (1985), found that college GPA and a general test 
of achievement were rather weak predictors of students' scores on the NTE science 
specialty tests. Elmore and Ellett (1978) found negative correlations among NTE scores 
and various measure of affective behavior. 

Tests of general scholastic abilit}' and tests of the teacher's knowledge base have 
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often been found to be highly correlated. (See, for example, studies by Egan & Ferre, 
1989; Lovelace & Martin, 1984; Tarver & Carr, 1983). In fact, these relationships are 
so notable that Lovelace and Martin (1984, p. ii) have suggested that **perhap$ students 
who score high on specific ACT subtests might be exempt from similar subtests on the 
revised NTE, thus saving students needless expense." 

What has not sufficiently emerged from these studies is consistency in research 
design. For example, NTE specialty ai*ea tests have alternately served as both 
independent (Erickson, 1971) and dependent (White & Tierney, 1989) variables. 
Research design has also often failed to address the issues of most practical significance 
(Leiser, 1981). As Moore, Schurr, and Henriksen (1991, p. 1024) noted, ''relatively few 
studies have evaluated correlations of the subtests with measures of teacher 
effectiveness/' Interestingly, when such studies have been conducted, scores on NTE 
subtests have served as rather weak predictors of scores on measures of teaching 
performance (e.g., Brown & Wells, 1988; Lovelace & Martin, 1984; Moore, Schurr, & 
Henriksen, 1991; Salzman, 1989), suggesting that measures of the potential teacher's 
knowledge base as measured by the NTE do not indicate who the more successful 
teachers will be. 

Purpose of the Present Study 
To date, relatively few studies have employed effective means for determining the 
individual contributions of various variables to ^;he prediction of teacher competence, 
and virtually none of the studies have used multivariate methods to assess the 
performance of the various measures of interest in light of their multivariate reality . 

i 
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Consequently, the purpose of the nresent study was to investigate the relationships 
among a host of variables used to measure teachet competence collected ou a sa?nple of 
student teachers at a given university using appropriate multivariate methods. 

Sample 

The sample utilized for the present study consisted of teacher education students 
(n ~ 496) enrolted at a comprehensive university in Mississippi. All 196 students had 
completed their academic program, including student teaching. Data utilized for the 
study were gathered from archival records housed in the universitj's records office and 
office of educational field experiences. Both computerized and paper records were 
utilized. Variables of interest inciudcd die following: 

(a) ACT scores— composite score as well as rcores on the English, math, social 
studies, and science subtests; 

(b) NTE scores— scores on edch of the four components of the NTE (general 
knowledge test^ communication skills test, professional knowledge test, and specialty area 
test). • 

(c) teaching performance scores— total scores on the Mississippi Teacher 
Assessment Instruments (MTAI) across two teaching situations and two raters (four 
evaluations per student) during the student teaching experience. The MTAI (Bureau of 
School Improvement, 1987) is purported to measure 14 teaching competencies across its 
42 continuous items, resulting in three subscaie scores, namely teaching plans and 
materials, position skills, and interpersonal skills. For purposes of the present study, the 
MTAI total scale scores were utHized since four different evaluations of the student were 
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utilized. Breaking down the total scale scores into the three subscales for each 
administration of the MTAI would have resulted in a very large and hard to manage set 
of variables. 

The typical person in the sample was 29 years old (mean age of 29; SD = 7.25), 
female (89.2%), Caucasian (96.5%), and relatively a *'good" student (mean GPA of 3.4; 
SD = .42). A wide variety of academic majors were represented, with elementary' 
education majors (n = 284) bdng the largest subgroup (57.3%) of the sample. Of the 
431 students for whom stt^dent teaching grades were available, 386 (89.6%) received As, 
39 (9.0%) received Bs, and 6 (1.4%) recch ed O. 

Method 

Once data were collected and coded, a canonical correlation analysis was 
performed using the SPSSx MANOVA procedure. By default, the MANOVA procedure 
conducts a cano&ilcal correlation analysis if no categorical independent variables are 
specified and if predictor variables for use in the canonical analysis are specified as 
covariates. Five variables were included in the teaching performance criterion variable 
set—total scores from the two MTAI evaluations by the university coordinator 
(MTAIUNIl and MTAIUNI2), total scores from the two MTAI evaluations by the 
cooperating teacher in the school(s) in which the subject did student teaching 
(MTAISCHI and MTAISCH2), and the student teaching fmal course grade (STGRA) 
received by the student. For STGRA, an A was assigned a value of 4, a B was assigned 
a value of 3, and a C was assigned a value of 2. The knowledge-base predictor variable 
set included four variables— the student's scores on the general knowledge (GKNTE), 
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communication skills (COMNTE), and profess&oual knowledge (PKNTE) portions of the 
NTE and the student^s overall university grade point average (GPA) based on a four 
point scale. The NTE specialty area test scok'es were not included In the analysis since 
students across different teaching majors took different area tests. 

A single null hypothesis ^vas proposed for investigation and subjected to empirical 
testing via the canonical correlat!,.. analysis: 

Teaching performance, as measured by four MTAI evaluations and student 
teaching outcome grade, >viil not be correlated at a statisticn.Hy significant 
(E " -05) level with teachers' knowledge base, as measured by the student's 
overall university GPA and the general kikowXcdge, communication skills, 
and professional knowledge subtests of the NTE. 

Results 

Intercorrelations among the four predictor and five criterion variables included in 
the canonical correlation! a.^alysis are presented in Table 1. Canonical correlation 
analysis creates weighted composites of the variables in each of the two variable sets and 
then computes a bivariate correlation between these two weighted composite variables. 
In essence, each variable is multiplied by a derived weight and then the weighted 
variables in each set are summed to produced a single "synthetic" variable. The 
bivariate correlation bet>veen these two variable composites is the canonical correlation 



INSERT TABLE 1 ABOUT HERE 



Evaluating Teacher Performance— p. 8 

For a given canonical correlation analysis, the number of canonical roots, or 
functions, yielded by the analysis is eq xal to the number of variables in the smaller 
variable set. Jn the present analysis, the predictor set included four variables and the 
criterion set included five variables; hence, the canonical analysis yielded four roots. 

The four canonical functions derived from the analysis are presented in Table 2. 
Based on the magnitude of the effect (R/) of the functions and c/^nsidering their levels of 
statistical significance, only the first function was interpreted. This first function yielded 
a canonical correlation (RJ of .490 (R<,^= .240; ja < .001), indicating that the academic 
performance predictor variables collectively accounted for approximately 24% of the 
variance in the scores on the five teaching performance variables. Consequently, the 
foregoing null hypothesis which stated that there would be no statistically significant 
relationship between the tvvo variable sets was rejected. 



IiNSERT TABLE 2 ABOUT HERE 
When a noteworthy canonical function is interpreted, it is generally appropriate 
to determine the degree to which various variables included in the analysis have 
contributed to the canonical function. Canonical correlation analyses yields two sets of 
weights that have been proposed for use for this purpose, i.e., canonical function 
coefficients and canonical structure coefficients. Function coefjicients are the derived 
weights applied to each of the variables in a given ^et in order to obtain the composite 
variate used in the canonical correlation analysis/ Even though the absolute magnitude 
of the function coefficients may be somewhat reliable in determining the contribution of 
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a variable to the composite, it has been shoiivn (e.g., Daniel, 1990; Thompson & Borrello, 
1985) that the numerical values of these coefTicients are highly affected by collinearity of 
variables in a given set as is often the case when employing canonical correlation. As 
shown in Table 1, variables included in the present analysis within the given sets have a 
relatively high degree of collinearity, suggesting that the funcnon vt^eights may not be 
reliable in indicating variable contributions. 

Canonical structure coefficients may also be used to suggest the amount of 
variable contribution to a canonical function. Structure coefficients indicate the degree 
of correlation of a given variable in the set with the canonical composite for the variable 
set. Structure coefficients tend to be much less susceptible to instability due to 
muiticollinearity of the variables in a given set. Hence, structure coefficients are 
generally considered as more reliable indicators of variable contribution. 

The canonical function and structure coefficients for the variables in the 
predictor and criterion sets are presented in Table 3. Since only the first function was 
worthy of interpretation, the structure coefficients for this root only were interpreted. 
These coefficients indicate that all of the variables in the criterion set made noteworthy 
contributions (all structure coefficients exceed ].70[) to the composite, suggesting the 
importance of further study of these variables. These rather large structure coefficients 
also serve as an indication of the high degree of collinearity among the variables in this 
set. 



INSERT TABLE 3 ABOUT HERE 
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Interestingly, only two of the predictor variables contributed to a substantial 
degree to the predictor variable composite, i.e.^GPA (structure coefflcient = -.937) and 
PKNTE (structure coefficient = -.359), with GPA making the most noteworthy 
contribution. Hence, it would appear that the general knowledge and coEnmuni(:ation 
portions of the NTE lack the ability to predict teaching performance of student teachers 
using total scores on the MTAI and student teaching grades as outcome variables. 

Discussion 

Few would debate the necessity of having quality teachers to staff America's 
schools; however, determining the most appropriate way to assure teacher quality is the 
source of much debate. In the present study two general sets of teacher competency 
measures were investigated, i,e., measures of the teacfier's knowledge base and measures 
of the teacher's performance in the classroom. The foregoing results suggest that the 
two sets of competencies are indeed related to a moderate degree. What is most 
disillusioning, however, is the fact that the subtests of the NTE are for the most part 
poor predictors of teaching behavior. These findingfi are consistent with those of 
previous research studies which suggest that the teacher's knowledge base, especially as 
measured by tests such as the NTE, is not very highly related to actual teacher 
perforraancw. By contrast, it would appear that GPA is a relatively good predictor of 
teaching performance. 

One possible explanation for these findings would be that good scores on the NTE 
arc a necessary but not a sufficient requirement for good teaching ability. lo other 
words, a person who could not obtain a minimal "cut" sc^ire on the NTE subtests would 

1 " 
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find it hard to be successful as a teacher, but not everyone who obtains above the cut 
score level will necessarily be a good teacher. One might further argue that by allowing 
only those students who have been successful at meeting a minimal performance level on 
one or more parts of the NTE to be allowed to continue in a teacher education program, 
a portion of the variance in the NTE scores has already been eliminated; hence, the 
correlations among NTE scores and other variables of interest will be somewhat 
diminished. This is not likely, however. GPA is often used as a screening device, and it 
appears to maintain its power as a predictor of teaching performance despite the fact 
that persons with lower CPAs are, in effect, removed from the sample early in the 
teacher screening process. 

Of the three NTE subtests included in the present study, it is interesting that the 
professional knowledge subtest made the greatest contribution to the canonical analysis. 
This result may indicate that more generalized academic functioning as measured by the 
general knowledge and communication skills subtests does not necessarily contribute to 
one's teaching ability, but that there is value m the professional "know-how" that 
teachers gain as part of participating in a teacher education program. This fmding may 
offer evidence that teaching has begun to develop a focused knowledge base on which 
effective teaching practices are developed. Ihe absence of such a knowledge base has 
often been cited as a reason for the lack of professional status of teaching. By contrast, 
the recognition of a commonly recognized body of professional knowledge which cau be 
shown to be related to appropriate teaching performance may do much to raise the 
occupational status of teaching. 
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Effective teaching requires many basic teacher competencies. None of these 
competencies" exists in a vacuum; in fact, the tezciter candidate is developing a host of 
these skills simultaneously. Moreover, there exists a complicated web of relationships 
among all of these skills as well as other facets of the prospective teacher's life 
experiences. The present study has demonstrated via multivariate methods a research 
design that is sensitive to these realities. Additional studies of this nature are needed to 
further investigate these complicated relationships. For example, one desirable strategy 
would be to break down effective teaching into a number of behaviors rather than using 
a global teaching performance score (e.g., the MTAI total score) as used in the present 
study. As noted previously, the MTAI total score can he further divided into three 
subscales measuring competencies related to (a) teaching plans and materials, (b) 
interpersonal skills, and (c) position skills, each of which could serve as a separate 
dependent variable in a multivariate analysis. Conducting further studies of this type 
would provide evidence regarding those variables which best correlate with teaching 
behaviors when considered in the light of other significant variables* 
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Table 1 
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( 

P-= 


.0273 
424) 
.288 


.0703 
( 430) 
P= .073 


1.0000 
( 0) 
P= . 


.6441 
( 473) 
P- .000 


.7240 
( 494) 
P' .000 


.2647 
( 461) 
P= .000 


PKNTE 


.1763 
( 429) 
P= .000 


.1468 
( 415) 
P* .001 


( 

p= 


.1412 
429) 
.002 


( 

P= 


.1219 
415) 
.006 


.1258 
( 422) 
P- .005 


.644: 
( 473) 
P= .000 


1.0000 
( 0) 
P» . 


.5897 
( 474) 
P- .000 


.4284 
( 442) 
P« .000 


GKNTE 


.0252 
( 438) 
P- .300 


♦.0259 
( 425) 
P= .298 


( 

P= 


.0054 
438) 
.455 


( 

P= 


.0611 
425) 
.104 


.0103 
( 431) 
.416 


.7240 
( 494) 
P- .000 


.5897 
( 474) 
P= .000 


1.0000 
( 0) 
P= . 


.2525 
( 462) 
P= .000 


GPA 


.3404 
( 408) 
P- .000 


3571 
( 396) 
P= .000 


( 

P' 


.4011 
408) 
,000 


( 

P- 


.4218 
396) 
.000 


.4038 
( 401) 
P- .000 


.2647 
( 461) 
P- .000 


.4284 
( 442) 
P= .000 


.2525 
( 462) 
P- .000 


1.0000 
( 0) 
P- . 
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Table 2 

Eigenvalues and Canonical Correlations 
Root No. Eigenvalue Pet. Om. Pet. Canon. Cor.Squared Cor. 



1 


.315 


90.473 


90.473 


.490 


.240 


2 


.026 


7.482 


55.955 


.159 


.025 


3 


.007 


1.910 


99.866 


.081 


.007 


4 


.000 


.134 


100.000 


.022 


.000 



BEST COPY AVAILABLE 

'0 
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Table 3 

Canonical f-unction and Structure Coefficients 
Standardized canonical function coefficients for DEPEJfflEKT variables 

Function Mo. 

Variable 



MTAISCHl 
MTAISCH2 
MTAIUNIl 
MTAIUNI2 
STGRA 



1 


2 


3 


4 


240 


.788 


1.004 


•1.127 


175 


.799 


.141 


1.266 


132 


-.434 


-.163 


1.313 


515 


.331 


•,919 


-1.195 


140 


'1.367 


.154 


-,215 



Structure coefficients for DEPENDENT variables 
Function No. 



Variable 

^frAISCHl 
MTAISCH2 
MTAIUNIl 
^frAIUNI2 
STGRA 



1 


2 


3 


4 


721 


.065 


.650 


-.149 


800 


.242 


-.036 


.394 


806 


-.196 


.254 


.154 


906 


.065 


-.401 


-.120 


817 


-.467 


.166 


.061 



Standardized canonical function coefficients for PREDICTOR variables 

Can. Var. 



Predictor 


1 


2 


3 


4 


COMNTE 


-.020 


-.383 


.416 


•1.498 


PKNTE 


-.227 


1.182 


.629 


,417 


GKNTE 


.450 


-.855 


.171 


1.161 


GPA 


-.967 


-.407 


-.307 


,113 


Structure coefficents for PREDICTOR variables 








Can. Var. 






Predictor 


1 


2 


3 


4 


COHNTE 


-.156 


-.366 


.854 


-.335 


PKNTE 


-.359 


.242 


.882 


.188 


GKNTE 


.019 


•.5?1 


.772 


.349 


GPA 


-.937 


-.296 


.135 


.125 



